Enforcing Monotonous Shape Growth or Shrinkage in Video Segmentation

One of the great challenges in computer vision is automatic segmentation of objects in videos. This task becomes more difficult when image sequences are subject to low signal-to-noise ratio or low contrast between intensities of neighboring structures. Such challenging data are acquired routinely, for example, in medical imaging or satellite remote sensing. While individual frames can be analyzed independently, temporal coherence in image sequences provides a lot of information not available for a single image. In this work, we focus on segmenting shapes which only grow or shrink in time, from sequences of extremely noisy images. We consider image sequences, where both foreground and background intensity distributions can vary significantly over time, foreground can be heavily occluded or undistinguishable from a part of the background. Most of previously-proposed spatio-temporal methods rely on coherence of foreground/background intensity distributions in consecutive image frames, and therefore fail when segmenting such noisy data sets. Few approaches have been designed for spatio-temporal segmentation of shapes from magnetic resonance (MR) images with low signalto-noise ratio [3, 4]. Applied to multi-temporal sequences that show a monotonously growing or shrinking structure, these smoothing methods bias results towards the mean shape obtained from averaging consecutive segmentations and, hence, underestimate rapid growth or shrinkage events. To address this issue, we propose a new segmentation framework based on graph cuts for the joint segmentation of an image sequence. It introduces growth or shrinkage constraint in graph cuts by using directed infinite links, connecting pixels at the same spatial locations in successive image frames. By minimizing an energy computed on the resulting spatio-temporal graph of the image set, the proposed method yields a globally optimal solution. Differently from the state-of-the-art spatiotemporal techniques, it does not rely on the coherence of the intensity in time, but only on the coherence of the shape. Graph cut is an optimization tool, which can be used to find the globally optimal binary segmentation of images [1], where the segmentation criterion E is related to a Markov Random Field with submodular interaction terms: